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ABSTRACT 

This document presents a brief summary of findings and 
actions over the last two years in three areas of Germanna Community 
College's overall student outcomes assessment program. First, program-placed 
students were tracked from developmental courses into subsequent collegiate 
coursework. While developmental student performance was roughly comparable to 
non-developmental students in subsequent English courses, completion rates in 
a transfer math course sequence were lower. Second, transfer data indicated 
that Germanna students performed equally well in senior college English and 
psychology courses compared to students who completed prerequisites at the 
senior college. However, lack of additional empirical data from Germanna ' s 
primary transfer destinations has impeded more extensive assessment of 
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(telecourses, compressed video and Web-based courses) was found to represent 
4% of total credit instruction during 1997-98, an increase after a decline in 
enrollment during 1996-97. The "non-completion rate" for telecourses in 
1997-98 was 17.5%, compared to a 9% rate for traditional on-campus courses. 
Only 38% of respondents who enrolled in the most popular compressed video 
course (Spanish) said they would recommend it to a friend as a compressed 
video course, while 76% would recommend it as a non-video course. (PGS) 
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Student Outcomes Assessment 1997-98 presents a brief summary of new findings and actions 
in three specific areas of Germanna Community College’s overall student outcomes assessment 
program, according to the VCCS reporting guidelines in effect since 1995. For the first two of 
these areas, which were last formally reported in 1996, the present report covers activities during 
the last two academic years. The final section responds to a “Topic of General State Interest,” and 
covers activities over an extended period. 

• Section II reports on assessment activities in developmental education, including 
evaluations of the course placement process, enrollment and persistence in 
developmental courses, and success in subsequent college-level coursework. 

• Section III reports on transfer student success, utilizing initial information from the 
“Course-Based Model of Transfer Success” project directed by Thomas Nelson 
Community College. 

• Section IV reports on the “Topic of General State Interest,” which is learning outcomes 
in technology-mediated instruction. The section compares completion rates and grades 
for courses offered in technology-based formats (telecourses, compressed video, and 
Web-based) with those offered in traditional formats. It also provides an initial student 
evaluation of instructional quality in compressed video courses. 

This summary report does not include data tables or supporting documentation, which are 

available from the' Office of Research and Planning. 
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Like the other Virginia community colleges, Germanna’s educational programs are available to all 
who may benefit from them — regardless of prior academic preparation. As a consequence of this 
policy, many of the students who enter Germanna are not academically prepared for collegiate 
coursework. They may have pursued a “vocational” program in high school, with or without the 
benefit of a Tech Prep college-oriented curriculum; they may have attended high school many 
years ago, or not at all; or they may have decided only later in their lives to attempt a post- 
secondary education. Other students are enrolling in order to improve their job-related skills after 
some time in the workplace, or in order to move up in a changing labor market. For all these 
students, developmental coursework provides an opportunity to refresh unused skills, or to fill 
gaps in previous academic preparation. Developmental courses prepare students for success in 
collegiate coursework. 

This section presents the results of an analysis of placement and developmental coursework at 
Germanna between 1994 and 1998. Data for this analysis were obtained through the VCCS 
Research and Assessment Data Support System (RADSS). The analysis describes the overall 
context for developmental coursework, as part of a comprehensive program of transfer, 
occupational, and general interest courses; identifies the patterns of student placement into 
developmental courses and actual enrollment in those courses; and tracks students from 
developmental courses into collegiate coursework, comparing their success in those classes with 
students who have not enrolled in developmental courses. 

A. New Findings 

Goal(s)/Objective(s) Being Assessed 

1. To assess the effectiveness of the placement process, whereby students are identified for 
developmental assistance to prepare them for success in college-level courses. 

2. To track the persistence of students identified for developmental coursework, as they enroll in 
and complete developmental courses, and as they make the transition from developmental to 
college-level courses. 

3. To evaluate the success of students completing developmental coursework in attempting 
college-level courses, in comparison with non-developmental students. 

Evaluation Method(s) 

1 . Analysis of data on the identification of students for developmental coursework (placement), 
and their subsequent enrollment in developmental math and English courses. 

2. Analysis of data on completion and persistence patterns for students enrolling in 
developmental courses, and then in subsequent college-level courses. 
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Findings, 1994-95 through 1997-98 

1 . Developmental courses comprised less than 6% of total credit enrollment during each of the 
last four academic years (1994-95 through 1997-98). However, among math offerings (MTH 
courses only) the proportion is much higher, at 39%. 

Identification for and enrollment in developmental coursework 

2. Earlier analyses ( Student Outcomes Assessment Report 1995-96) included all students 
enrolled in a given term, and calculated a proportion of approximately 60% who were 
identified as requiring developmental coursework. However, each semester a substantial 
proportion of Germanna students are enrolled in classes for personal development, and are 
not pursuing any degree or certificate program. These students are not required to complete 
placement testing (or provide SAT scores) and therefore frequently have not been evaluated 
in terms of their preparation for college-level English and math courses. In order to give a 
more accurate picture of which students are identified for developmental coursework, 
therefore, the present analysis includes only program-placed students. 

3. From Summer 1994 through preliminary enrollment for Fall 1998, 6,164 students were 
enrolled in a degree or certificate program. Approximately 77% (4,726) of these students had 
placement scores on file. 

4. Of the students with placement scores, approximately 75% (3,565 of 4,726) were identified 
for developmental coursework. Of these, 934 (26.2%) were recommended for developmental 
English, and 3,372 (94.7%) were recommended for developmental math. (These proportions 
include some students who were recommended for both.) 

5. Approximately 60% of the 3,565 students identified for developmental coursework had 
enrolled in developmental courses by the Fall of 1998. This proportion varies between 
students recommended for developmental English (63% enrolled) and math (58%), but 
especially according, to the student’s curriculum. . Most of the students identified for- 
developmental courses who were enrolled in transfer degree programs (69%) had already 
taken developmental courses, while only 47% of students in occupational degree programs 
and 34% of students in certificate programs had enrolled. 

6. It should also be noted that, among those students for whom developmental courses were not 
recommended, some 1 5% chose to enroll in developmental courses on their own initiative. 

Developmental course outcomes 

7. Over the period covered by this analysis, approximately 64% of the students who enrolled in 
developmental courses completed them satisfactorily. (Standard grades are not assigned in 
developmental courses.) This proportion applies generally to both English and math courses, 
and does not vary substantially between those who were recommended for developmental 
coursework, and those who chose the developmental course despite initial placement at the 
college level. 

Further, two different measures are available for examining the success of developmental course 

completers in subsequent college-level courses: 
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8. In English 111 “College Composition I,” among program-placed students who were 
identified for developmental English, the students who enrolled in developmental courses 
achieved essentially the same average grade as those who did not enroll in the developmental 
course (GPA 2.37 vs. 2.38) and were slightly more likely to complete ENG 1 1 1 successfully 
(72% earning a ‘C’ or better, compared to 67%). 

9. A slightly different comparison between students who completed the developmental writing 
course (ENG 01) and those who did not take any developmental course also shows that the 
developmental completers achieved similar results: 70% of the students who completed ENG 
01 successfully achieved at least a ‘C’ in ENG 111, compared with 74% of non- 
developmental students, although the latter did achieve higher average grades (GPA 2.73 vs. 
2.36). 

10. Among students taking courses in the transfer math sequence (MTH 151, 163, 173, or 240) 
there is a clearer gap between students enrolling in developmental courses and those who do 
not. Among program-placed students identified for developmental math, a lesser proportion 
of the students who did enroll in developmental courses completed col lege- level math 
courses successfully (71% earning a ‘C’ or better) than among the identified students who did 
not enroll in developmental courses (81% of whom achieved at least a ‘C’). 

1 1 .However, when developmental course completers are compared with students taking college- 
level math who did not take a developmental course, the gap is somewhat narrower. 
Approximately 75% of the students who completed MTH 03, 04, or 05 subsequently 
achieved at least a ‘C’ in the college-level math courses listed above. The comparable 
proportion among non-developmental students in those courses was 79%, and the latter also 
earned slightly higher average grades (2.76 vs 2.66). While these differences attain statistical 
significance, they do show that developmental completers perform comparably to their non- 
develnnmental classmates 

1 v 



B. Use of results 

As part of the ongoing review of the placement and developmental education process, faculty in 
communications and humanities disciplines recommended two changes which have been 
implemented for 1997-98: 

• Offering ENG 06 “Reading in the Content Area” as a co-requisite for students with 
“borderline” results on the reading placement test. 

• The developmental placement thresholds for reading were raised across the board: for referral 
to local Adult Basic Education programs, placement in developmental courses, and for 
“unrestricted” placement in content courses. 

C. Problems in assessing developmental education, and proposed solutions 

• Faculty members continue to question the validity of the COMPASS exam for math 
placement, as currently administered. There is a concern about the “adaptive” nature of the 
test, and whether students truly understand how this form of testing works. 
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• Faculty also have concerns about other aspects of placement testing (e.g., the testing 
environment and students’ lack of preparation for testing) and the advising procedure (e.g., 
availability of placement scores in a timely fashion and the need to integrate additional high 
school background information into advising). 

• It appears that a significant, although undetermined, proportion of students are shifted to 
courses different from their initial placement on the basis of placement tests. Faculty in both 
English and math courses frequently use their own initial diagnostics, and advise students 
who appear to be incorrectly placed to move to more appropriate courses. These changes are 
not recorded in the placement data available through SIS and RADSS for later analysis. 

• The College’s Teaching and Learning Committee, in collaboration with the faculty of the 
Communications and Humanities discipline cluster, undertook a review of the placement 
process during 1997-98. The review included informal surveys of faculty regarding the 
process, and collection of data from other VCCS colleges. The initial review produced 
changes for 1998-99 as noted in (B) above. The Teaching and Learning Committee is 
charged with the completion of a more formal review during 1998-99. 

D. Exemplary processes 

While the task sometimes appears daunting, Germanna faculty and staff continue to examine the 
placement and developmental education processes, utilizing a mixture of concrete empirical data 
and anecdotal observations. Significant progress has been made during the last two years, and is a 
continuing goal for 1998-99. 

E. Self-evaluation of Progress 

Germanna is making satisfactory progress in assessing developmental education. 
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A. New Findings 

As of this writing, the only new data available on the success of Germanna’s students in 
transferring to baccalaureate colleges and universities has been provided by Thomas Nelson 
Community College through its Course-Based Model of Transfer Success (CBMTS) project. This 
first section summarizes the information provided in the CBMTS report for Germanna (April 
1998). 

Goal(s)/Objective(s) Being Assessed 

• To evaluate the success of students who complete prerequisite courses at Germanna in 
attempting upper-division courses at transfer institutions. 

Evaluation Method(s) 

• CBMTS tabular data for 1993-94 through 1996-97, showing completion of specific courses 
at six transfer institutions for three groups: students who completed prerequisites at the 
transfer institution (“native” students); students who completed prerequisites at Germanna; 
and students who completed prerequisites at other VCCS colleges. 

Findings 

1 . The data is presented as a table of results in specific upper-level courses at six colleges over a 
period of four years, and therefore compiles tens of thousands of course completions. Despite 
this large amount of data, however, the tables generally provide information for less than ten 
Germanna students in any given course, which is an insignificant basis for valid statistical 
comparisons. Aggregated comparisons at the discipline level are also not directly useful, as 
different upper-division courses in the same discipline have different prerequisites. The data 
must therefore be transformed. 

2. Using crosswalk tables provided with the report, it is possible to identify the transfer college 
courses which have a common prerequisite. The data can then be re-tabulated to produce 
meaningful information. The information presently available provides sufficient outcomes 
information on two course areas: courses for which ENG 111-112 is a prerequisite, and 
courses for which PSY 201-202 is a prerequisite. 

3. In English courses, students who completed their prerequisites at Germanna performed 
equally well with those who completed prerequisites at the transfer institutions. Of the 
Germanna students, 84.4% attained at least a ‘C’ or a ‘Pass’ in the target courses, compared 
with 85.7% of the students completing prerequisites at the senior college. A slightly higher 
proportion of students who completed prerequisites at other VCCS colleges (88.0%) was 
successful. These overall differences are not substantial. 
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4. Among the three institutions which produced sufficient data for individual comparisons, only 
George Mason University showed a different pattern. Of students who completed their 
English prerequisites at GMU, 93.3% completed the upper-division courses successfully. 
Among other VCCS college students, the proportion was 92.3%. The proportion of 
Germanna students — although still quite substantial — was relatively lower, at 84.0%. 

5. In psychology courses, Germanna students again performed comparably well with the other 
groups. Students with Germanna prerequisites completed the upper-level courses successfully 
in 80.0% of the tabulated cases, while the comparable proportions for “native” students and 
students from other VCCS colleges were 84.3% and 83.2%, respectively. 

6. Only two colleges produced sufficient numbers for individual comparisons: At Old 
Dominion University, the Germanna success rate was slightly lower than that for the other 
two groups; at Radford University, the Germanna success rate was slightly higher. These 
differences are quite small, however, and should not be exaggerated. 

B. Use of results 

The analysis of the CBMTS data summarized above has been shared with faculty members in the 
relevant disciplines, and provides a useful initial basis for a discussion of specific transfer 
outcomes. However, the data as presently constituted are limited in two respects: they do not 
include results from Germanna’ s primary transfer destination (Mary Washington College); and 
they are presented in a format which makes interpretation difficult. The Assessment Coordinator 
is pursuing further analysis within the context of this project. 

Although the lack of additional empirical data on student success — especially from Germanna’s 
primary transfer destinations — has precluded program-specific improvements, the College has 
undertaken continued efforts to improve the quality of its preparation for transfer and the process 
of transfer advising: 

• Following a recommendation arising from the SACS accreditation process, the College’s 
Curriculum Committee developed a list of approved electives for transfer degree programs. 
Courses on the list meet a twofold standard: they have been accepted for transfer by Virginia 
public colleges and universities, and they are taught by appropriately qualified faculty. 

• Germanna’s Counseling unit has expanded the amount of transfer-related information 
available to students. A significant new source of transfer information is College Source, an 
online resource which provides access to 7,000 college catalogs. 

C. Problems in assessing transfer student success, and proposed solutions 

The greatest obstacle to a complete assessment of the success of Germanna’s transfer students 
remains the difficulty in obtaining data from senior institutions. Since the 1996 analysis, which 
was based on improved reporting from transfer colleges for that year, only two of Germanna’s 
five primary transfer institutions have provided any new data. Further, the following difficulties 
with the quality of the reported data remain: 
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• Format - several institutions still do not submit data in the format approved by the Virginia 
Assessment Group and SCHEV. Some utilize different file formats (e.g. spreadsheet files), 
some provide inconsistent data record layouts, and some include different coding for 
“standard” variables. Although these differences are not insurmountable, their cumulative 
effect is to make the analysis more cumbersome — and likely less accurate. 

• Completeness - Most data records do not include the actual academic standing variable — 
although many institutions ostensibly create a location for the information in the dataset, and 
others provide a “default” code for this variable. Several colleges do not provide student- 
identifiable information on non-enrollees, which creates a serious weakness in the analysis of 
admission and enrollment rates. 

• Follow-through - The colleges which have reported, with one or two exceptions, do not 
provide information for more than one year following transfer. 

The direct exchange of data between transfer colleges and Germanna has not been successful to 
this point. Although expanded efforts on Germanna’ s part to make contact might well bring better 
results, it does appear that a duplication of effort is taking place. Our recommendation is that 
SCHEV take the lead in releasing individually-identifiable data on transfer success to the 
community colleges. 

Germanna will also collaborate with Thomas Nelson Community College to pursue additional 
analysis from the CBMTS project, and supports continued funding for that effort. 

D. Exemplary processes 

Although the College’s efforts to date in assessing transfer student success are satisfactory, they 
do not rate a label of “exemplary.” 

E. Self-evaluation of Progress - 

Although some progress has been made since this topic was last reported in 1996, the College’s 
progress in assessing transfer student success is best characterized as “uneven”. This is due 
primarily to a lack of the data necessary for a complete analysis, as discussed in (C) above. 
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Section IV. 

Topic of General State Interest: Technology-mediated Learning 

Germanna expanded its instructional offerings to include technology-based formats with the 
addition of telecourses during 1994-95. The telecourses have been broadcast over local cable 
television stations and have also been available on videotapes. For the 1997-98 academic year, 
the College added courses delivered through compressed video to the Commonwealth classrooms 
at both campuses. These courses included both offerings transmitted between the campuses and 
courses offered from other VCCS colleges. In Spring 1998, Germanna also offered a Web-based 
psychology course on a pilot basis. 



A. New Findings 

Goal(s)/Objective(s) Being Assessed 

• To determine whether student retention and success rates for technology-mediated 
instruction are comparable to those for other modes of instructional delivery. 

• To identify the appropriate audience(s) for technology-mediated instruction. 

• To identify specific aspects of technology-based instructional delivery which may present 
barriers to student learning, and to remove those barriers wherever possible. 



Evaluation Method(s) 

1 . Comparison of course completion and withdrawal rates by mode of instruction. 
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indication of the relative success of students enrolled in alternative format courses. 



3. Comparison of student demographics by mode of instruction, as a means of determining the 
“target population” for technology-mediated instruction. 

4. Student evaluation of instruction, with specific components which address aspects of 
technology-based instructional delivery that may present barriers to student learning. 



Findings 

• Following a decline during 1996-97, enrollment in telecourses increased substantially for 
1997-98. Enrollment in all technology-based courses — telecourses, compressed video, and 
Web-based — comprised 4% (71.5 AFTES) of total credit instruction during 1997-98. 

• A previous report on completion patterns for students enrolled in telecourses (“An 
Assessment of Distance Learning Activities Delivered in Telecourse Format,” June 1996, 
summarized in the 1996-97 Assessment Report) noted the comparatively high rate of 
withdrawals and incompletes among telecourse students. The report recommended several 
changes in telecourses, which were implemented beginning in Fall 1996. This resulted in a 
decrease in the “non-completion rate” from 18.4% in 1994-95 to 9.3% in 1996-97. However, 
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this same rate returned to its previously high level for 1997-98, at 17.5%. (The comparable 
rate for traditional on-campus courses was approximately 9% throughout the period.) 

• Average grades in telecourses over the four-year period 1994-95 through 1997-98 did not 
differ significantly from grades in on-campus courses (GPA 2.88 vs. 2.84). However, it is 
worth noting that telecourse grade averages rose substantially during the period, from 2.44 in 
1994-95 to 3.01 in 1997-98. This is due in part to the improved completion rates noted 
above, but still bears further analysis. Average grades in compressed video and Web-based 
courses during 1997-98 matched the on-campus average exactly at 2.84. 

• Because the number of offerings has remained relatively small, the demographic 
composition of enrollment in technology-mediated courses has been significantly influenced 
by changes in specific course offerings from one semester to another. During 1997-98, the 
only year in which all current forms of instructional technology were employed, there were 
only three consistent differences discernible between technology-mediated and traditional 
on-campus courses: In telecourses, more of the students were female, and more were 
enrolled in transfer degree programs; in compressed video courses, students were slightly 
younger on average. Enrollment in the single Web-based course did not provide a basis for a 
demographic comparison. 

A specifically revised student evaluation of instruction was completed in all six sections of 
Spanish taught through compressed video between Germanna’s two campuses in Fall 1997. 
(Spanish courses comprised the largest single component of technology-based instruction during 
the year.) The evaluation produced the following results: 

• A majority of students (70%) would have preferred to take the course without utilizing 
compressed video. However, more students prefer to take the compressed video course than 
would be willing to travel to the opposite campus. Although this point may be essentially 
moot, it does indicate that the medium created some type of barrier for the students. 

• In what is perhaps the key finding of the evaluation, two-thirds of the students (67%) stated 
that they had been less willing to participate in discussions as part of the video course than 
they would have been in a “standard classroom” course. 

• The key difficulty in the video courses seemed to be the audio component, as 55% of 
respondents rated their ability to hear the instructor or students at the opposite campus as 
only “Fair” or “Poor”. A smaller proportion (38%) gave negative ratings to the video quality, 
and a majority (54%) felt that the course was appropriate for the medium. 

• The vast majority of students gave the instructor positive marks for availability (78%), 
timely return of graded materials (95%), course outline (95%), course notes (92%), 
relevance of assignments (87%), and adherence to course objectives (94%). 

• In summary, only 38% of respondents would recommend the course to a friend as a 
compressed video course, while 76% would recommend it as a non-video course. 
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Changes resulting from assessment findings 

1 . The revised student evaluation of instruction will be utilized in all courses taught through 
compressed video beginning with Fall 1998. 

2. The evaluation instrument developed for compressed video instruction will be revised for use 
in Web-based courses. 

3. It is imperative that the College develop direct assessments of student learning in technology- 
mediated courses which addresses the acquisition of specific skills or knowledge. Where 
possible, this assessment should provide a basis of comparison with the same course offered 
in a different format, as has been done previously for off-campus and dual enrollment 
courses. 

4. Policies and procedures for telecourses, which were developed as part of a “Strategic Plan for 
Distance Learning Activities” in 1996, are being utilized more systematically again for 1998- 
99. The “Strategic Plan” document will be revised to incorporate policies and procedures for 
compressed video and Web-based courses. 

B. Self-evaluation of Progress 

Germanna is making satisfactory progress in developing an assessment program for technology- 
mediated learning. 
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